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(54) Using database query technology for message subscriptions in messaging systems 



(57) The present invention relates to messaging 
systems in general and message broker systems in 
specific. More particularly, the invention relates to 
means and a method for processing message publica- 
tion requests and message subscription requests. 

The current invention transforms the subscription 
problem into a database query problem allowing to use 
query techniques known and optimized for many years. 
The basic approach of the current invention is to invert 
the relationship of message publication requests 
(treated as database data) and message subscription 
requests (treated as database queries) according the 
state of the art. Thus the core of the current invention is 
to treat incoming messages publication requests as 
database queries and to treat message subscription 
requests as database queries. 
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Description 

1 Background of the Invention 

1 .1 Held of the invention 

[0001] The present invention relates to messaging 
systems in general and message brokers in specific. 
More particularly, the invention relates to means and a 
method for processing message publication requests 
and message subscription requests. 

1.2 Description and Disadvantages of Prior Art 

[0002] One source of technology the current invention 
is based on is the database technology. For a thorough 
outline of the principles and concepts of database tech- 
nology refer for instance to R. Elmasri and S.B. Nav- 
athe, Fundamentals of database systems, 1989, 
Benjamin/Cummings. Of specific importance to the cur- 
rent invention is the fundamental concept of a query of 
database technology. A query is a request to a data- 
base system to retrieve from a database a collection of 
data qualifying under the query; this collection is called 
the answer set of the query. Typically, the database con- 
tains data about real-world entities like persons, orders, 
appointments etc.. A query is a prescription for select- 
ing subsets from these entities, possibly combine some 
of them, and present them in a certain form. 
[0003] Another source of technology the current 
invention is based on is the technology messaging sys- 
tems or message brokers in specific. A messaging sys- 
tem or a message broker is like a hub where messages 
are streaming in, are transformed, and are streaming 
out. Messaging systems offer the passing of messages 
through a store-store-and-forward funcionality. Mes- 
sage brokers in addition also allow to reduce the con- 
nection costs by supporting the many-to-many 
communication principle comprising the one "source- 
message" vs. multiple "sink-message" communication 
case. Compared to messaging systems a message bro- 
ker might be augmented by related services like: a mes- 
sage transformer, flow controller, message warehouse, 
message dictionary and management facility. 
[0004] The messages which are streaming into the 
messaging system/broker are referred to as being pub- 
lished. Messages which are streaming out of the mes- 
saging system/broker are referred to as being 
subscribed. A publication request specifies a message 
a certain application intends to offer to other interested 
applications, i.e. to subscribers. A subscription request 
specifies the subset of all incoming messages a partic- 
ular application is interested in, and the format in which 
it has to be presented to the subscriber. An overview on 
messaging systems and message brokers is given in R. 
Schulte, Message brokers: A focussed approach to 
application integration, Gartner Group, Strategic Analy- 
sis Report SSA R-401-102, 1996 as well as in "Enter- 



prise application integration and information flow", The 
Yankee Group - Internet Computing Strategies (Boston 
MA. 1997). 

[0005] Thus, a subscription request is similar to what 
5 has been described above as a "query" in terms of data- 
base technology. And exactly in this sense current state 
of the art messaging systems and message brokers do 
treat subscription requests. Expressed in database 
terms the "messages" are treated as "database data" 
10 and the "subscriptions request" are treated as "query" 
operating on that data filtering out the subscribed mes- 
sages; i.e. based on its specification messages are fil- 
tered, transformed and delivered to its requester. State- 
of-the-art implementations (for instance the Gryphon 
15 approach of IBM Watson Research) use techniques like 
variants of binary trees to represent subscription 
requests and process incoming messages accordingly 
for filtering. 

[0006] So far specific implementations for messaging 
20 systems and message brokers of these administrations 
and filtering tasks of message publication and subscrip- 
tion requests have been required. No exploitation of 
general purpose technologies delivering performant 
solutions has been possible so far. 
25 [0007] The message broker paradigm is about to gain 
increased importance in the area of application integra- 
tion. Its implementation therefore has to be further 
improved to be able to support the huge diversity of 
application needs. 

30 

1 .3 Objective of the Invention 

[0008] The invention is based on the objective to 
improve the processing of message publication 
35 requests and message subscription requests within 
messaging systems in general and message brokers in 
specific. 

2 Summary and Advantages of the invention 

40 

[0009] The objective of the invention is solved by the 
independent claims. 

[001 0] The objective of the invention is solved by claim 
1 . A messaging system is taught comprising first receiv- 
es ing means for receiving a message publication request 
comprising a message to be published. Said messaging 
system comprising second receiving means for receiv- 
ing one or a multitude of message subscription requests 
comprising a query on a message qualifying under said 
so query. Further a database (301) is comprised. Said sec- 
ond receiving means treats said message subscription 
request as database data and stores said message 
subscription request to said database (301). Said first 
receiving means treats said message publication 
55 request as a database query (302) and retrieves from 
said database a collection (303) of message subscrip- 
tion requests qualifying under said database query. 
[001 1 ] The technique proposed by the current inven- 
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tion makes database technology available to messaging 
systems. In database environments a query operates 
on a large amount of data, but in a message broker 
environment a large amount of queries (corresponding 
to a message subscription) operate on a single data 5 
item (corresponding to a publication of a message). Due 
to this circumstances database technology could not be 
exploited in a performant manner by messaging sys- 
tems. Moreover the current teaching significantly simpli- 
fies the implementation of messaging systems as an 10 
exploitation of the mature, sophisticated and high per- 
formant database technology is possible; no implemen- 
tation specific to messaging systems is required. 
Through the transformation of the subscription problem 
into a database query problem the current invention is 
allows to use query techniques known and optimized for 
many years. All advantages and benefits provided by 
database technology are immediately available to mes- 
saging systems, like persistency, recovery features, 
transactional behavior and many more. For a broad 20 
spectrum of business problems, no new algorithms 
must be invented and existing implementations can be 
reused. All these advantages are of specific value within 
messages systems required to provide high reliability 
against data losses and high availability. 25 
[0012] Additional advantages are accomplished by 
claim 2. According to a further embodiment of the pro- 
posed invention said second receiving means trans- 
forms a message subscription request into a database 
data representation. 30 
[001 3] By explicitly transforming a message subscrip- 
tion into a specific data representation the efficiency (for 
instance in terms of the amount of data to be stored or 
in terms of performance to process a later query for that 
data etc.) of the current approach can be increased fur- 35 
ther. 

[0014] Additional advantages are accomplished by 
claim 3. According to a further embodiment of the pro- 
posed invention said first receiving means transform 
said message publication request into a database 40 
query. 

[001 5] Through such a transformation, which may be 
adjusted to the transformation of the subscription 
according to claim 2, into a specific query representa- 
tion the efficiency of the current approach can be 45 
increased further. 

[0016] Additional advantages are accomplished by 
claim 4. According to a further embodiment of the pro- 
posed invention said first receiving means store said 
message into said database or into a separate message so 
database. 

[0017] Using database technology also for the mes- 
sages allows to profit from the database technology 
benefits for the message request too; examples are: the 
recovery features after system crashes, atomicity, integ- ss 
rity etc.. Moreover through this feature the door to mes- 
sage warehousing is opened. 

[0018] Additional advantages are accomplished by 



claim 5. According to a further embodiment of the pro- 
posed invention the current teaching is applied to a 
message broker and/or a message warehouse. 
[0019] Through its universality the current teaching 
may be applied in various areas. 
[0020] Additional advantages are accomplished by 
claim 6. 

[0021 ] According to a further embodiment of the pro- 
posed invention said database may be stored in main 
memory and/or on persistent memory. 
[0022] Again this demonstrates the flexibility of the 
teaching; messaging systems, for which the response 
time behavior is the primary requirement and which will 
store the database within main memory, as well as mes- 
sages systems, for which reliability against data losses 
is the primary requirement and which will store the data- 
base on persistent memory, may exploit the proposed 
technology. 

[0023] Additional advantages are accomplished by 
daim 7. According to a further embodiment of the pro- 
posed invention said database may be a relational data- 
base or a hierarchical database or a network database 
or an object-oriented database or a collection of flat 
files. 

[0024] Thus the current teaching is applicable to any 
type of database. The teaching does not limit the 
exploiter to a certain database technology; instead 
other criteria, for instance maturity, availability of the 
database technology or the type of data to be stored 
within the database, may decide on the database type 
to be selected. 

[0025] Additional advantages are accomplished by 
claim 8. According to a further embodiment of the pro- 
posed invention the messaging system is comprising 
sending means for sending said message to subscrib- 
ers of said collection of message subscription requests. 
[0026] Thus the message comprised in the message 
publication request and serving as the database query 
can finally be sent to all subscribers of that particular 
message or that message type. 

[0027] The objective of the invention is solved by claim 
9. 

[0028] A method of processing a message publication 
request is being taught comprising a message to be 
published, and one or a multitude of message-subscrip- 
tion-requests comprising a query on a message qualify- 
ing under said query. Said method comprises the 
following steps: 1) a first step of treating said message 
subscription request as database data and storing said 
message subscription request to said database (301), 
and 2) a second step of treating said message publica- 
tion request as a database query (302) and retrieving 
from said database a collection (303) of message sub- 
scription requests qualifying under said database- 
query. 

[0029] Benefits related to this teaching have been dis- 
cussed together with claim 1 above. 
[0030] Additional advantages are accomplished by 
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claim 10. According to a further embodiment of the pro- 
posed invention said first step is preceded by a sub- 
scription transformation step wherein said message 
subscription request is transformed into a database 
data representation. 

[0031 ] Benefits related to this teaching have been dis- 
cussed together with claim 2 above. 
[0032] Additional advantages are accomplished by 
claim 1 1 . According to a further embodiment of the pro- 
posed invention said second step is preceded by a mes- 
sage transformation step wherein said message 
publication request is transformed into a database 

query. 

[0033] Benefits related to this teaching have been dis- 
cussed together with claim 3 above. 
[0034] Additional advantages are accomplished by 
claim 12. According to a further embodiment of the pro- 
posed invention said first step stores said message into 
said database or into a separate message database. 
[0035] Benef its related to this teaching have been dis- 
cussed together with claim 4 above. 
[0036] Additional advantages are accomplished by 
claim 1 3. According to a further embodiment of the pro- 
posed invention said method is executed by a message 
broker and/or wherein said database is part of a mes- 
sage warehouse. 

[0037] Benefits related to this teaching have been dis- 
cussed together with claim 5 above. 
[0038] Additional advantages are accomplished by 
claim 14. According to a further embodiment of the pro- 
posed invention said method is storing said database in 
main memory and/or on persistent memory. 
[0039] Benefits related to this teaching have been dis- 
cussed together with claim 6 above. 
[0040] Additional advantages are accomplished by 
claim 15. According to a further embodiment of the pro- 
posed invention said database is a relational database 
or a hierarchical database or a network database or an 
object-oriented database. 

[0041 ] Benefits related to this teaching have been dis- 
cussed together with claim 7 above. 
[0042] Additional advantages are accomplished by 
claim 1 6. According to a further embodiment of the pro- 
posed invention said method is further comprising a 
third step of sending said message to subscribers of 
said collection of message subscription requests. 
[0043] Benefits related to this teaching have been dis- 
cussed together with claim 8 above. 

3 Brief Description of the Drawings 

[0044] 

Figure 1 is a diagram reflecting the query concept 
according database technology 

Figure 2 visualizes the relationship and the handling 
of message publication requests and mes- 
sage subscription requests according to 



the current state of the art of messaging 
systems 

Figure 3 depicts the relationship and the handling of 
message publication requests and mes- 
5 sage subscription requests according to 

the current invention and applicability of 
database technology 

4 Description of the Preferred Embodiment 

10 

[0045] If the current specification is referring to a data- 
base this term includes any type of database system, 

like-for_example-relational databases,hierarchical-data^ 

base, network databases or object-oriented databases; 
is the invention is not limited to a specific database system 
type. Moreover for the sake of the current invention the 
data model of the database system is irrelevant. Also, 
the database system might manage persistent data on 
disk and/or might manage data in main memory (i.e. a 
20 main memory database). 

[0046] This teaching does not impose any limitations 
to the nature of a message or the type of subscription. 
Moreover the term "message" is to be understood from 
a logical perspective. The actual communication mid- 
25 dleware may transfer the message using the connec- 
tion-oriented or the connectionless paradigm; the 
invention is not limited to any of these paradigms. 
[0047] The current invention relates to messaging sys- 
tems as well as message brokers. Therefore if features 
30 of the invention are described within this specification in 
the context of message brokers only the teaching may 
also be applied to messaging systems and vice versa. 

4.1 Introduction 

35 

[0048] Of specific importance to the current invention 
is the fundamental concept of a query of database 
technology, which is depicted in Figure 1 . A query (101) 
is a request to a database system to retrieve from a 

40 database (1 02) a collection of data qualifying under the 
query; this collection is called the answer set (103) of 
the query. Typically the database contains data about 
real-world entities like persons, orders, appointments 
etc.. A query is a prescription for selecting subsets from 

45 these entities, possibly combine some of them, and 
present them in a certain form. 
[0049] A messaging system or a message broker is 
like a hub where messages are streaming in, are trans* 
formed, and are streaming out. Messaging systems 

so offer the passing of messages through a store-store- 
and-forward funcionalrty. Whereas message brokers in 
addition also allow to reduce the connection costs by 
supporting the many-to-many communication principle. 
The messages which are streaming into the messaging 

55 system/broker are referred to as being published. Mes- 
sages which are streaming out of the messaging sys- 
tem/broker are referred to as being subscribed. A 
publication request specifies a message a certain 
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application intends to offer to other interested applica- 
tions, i.e. to subscribers. A subscription request spec- 
ifies ttie subset of all incoming messages a particular 
application is interested in, and the format in which it 
has to be presented to the subscriber. Message brokers 
also may support message dictionaries to hold meta- 
data descriptions of messages for development pur- 
poses for instance, and administration and monitoring 
tools to manage the operation of the broker configura- 
tion. 

[0050] For example, various stock exchanges might 
publish stock data periodically, i.e. the stock data is 
send to the message broker. Each stock exchange 
might use a different format to transfer its data. A sub- 
scriber has registered to all messages about IBM stock 
data exceeding $103 and requests its delivery in XML 
(extended Markup Language) format. 
[0051 ] Thus, a subscription request is similar to what 
has been described above as a "query" in terms of data- 
base technology. And exactly in this sense current state 
of the art messaging systems and message brokers do 
treat subscription requests. A visualization of this state 
of the art is given in Figure 2. Expressed in database 
terms the "messages" (202) are treated as "database 
data" and the "subscriptions request" (201) are treated 
as "query" operating on that data filtering out the sub- 
scribed messages (203); i.e. based on its specification 
messages are filtered, transformed and delivered to its 
requester. State-of-the-art implementations (for 
instance the Gryphon approach of IBM Watson 
Research) use techniques like variants of binary trees 
to represent subscription requests and process incom- 
ing messages accordingly for filtering. 
[0052] Messaging systems and/or mesage broker 
support request/reply and/or event-based communica- 
tion depending on whether the communication is initi- 
ated by the a source or a sink application. A sink asks 
the system for data and the system fulfills the request. 
In event-based communication communication is 
source driven. The message is generated by the source 
and passed to the system for relaying to the sinks. The 
current invention applies to both communication mod- 
els. 

4.2 Further Analysis of the Publish-Subscribe Prob- 
lem Area 

[0053] The following two problem areas can be identi- 
fied and are addressed by the current invention. 

4.2.1 Many-Queries-Single-Data-htem Characteristic 

[0054] Within the are of messaging systems typically 
many applications have subscribed with a message bro- 
ker corresponding to many different queries which have 
to be processed for each incoming message; i.e. mes- 
sage subscriptions requests correspond to database 
queries. Thus, the situation is the opposite as in data- 



base environments: In the latter environments a query 
operates on a large amount of data, but in a message 
broker environment a large amount of queries operate 
on a single data item. Extensive caching of messages 

5 can be helpful here, but even if applied to the current sit- 
uation, the fundamental characteristics, "relatively" 
small number of messages (database data) versus "rel- 
atively" large number of message subscriptions (data- 
base queries), is not reversed. Current query 

10 processing algorithms within messaging systems do not 
address this situation, thus, new algorithms must be 
invented or algorithms from other areas must be applied 
in this situation. 

is 4.2.2 Subscriptions In Message Warehousing 

[0055] In another class of message broker applica- 
tions the storage of published messages is required 
resulting in a so-called message warehouse. In this sit- 

20 uation, performing the filtering outside of the database 
as before seems to be unnatural since the database 
functionality is already part of the overall environment. 
[0056] A first step for improvement in this problem 
area is to store the message in the database before sat- 

25 isfying the subscriptions. Subscriptions will then be 
transformed into usual database queries and the que- 
ries are run against the database to retrieve all corre- 
sponding messages. This technique might work fine in 
certain scenarios: For example, if subscriptions do not 

30 have to be considered in real-time the broker can peri- 
odically work on the subscriptions, i.e. when a bulk of 
new messages has been materialized in the database. 

4.3 Inverting the Relationship of Message Publica- 
35 tion and Subscription Requests 

[0057] The current invention transforms the subscrip- 
tion problem into a database query problem allowing to 
use query techniques known and optimized for many 

40 years. Even when messages are not made persistent 
on disk, query technologies according the current 
teaching can be applied by considering messages and 
subscriptions as data in main memory databases. 
[0058] The basic approach of the current invention is 

45 to invert the relationship of message publication 
requests (treated as database data) and message sub- 
scription requests (treated as database queries). Thus 
the core of the current invention is to treat incoming 
message publication requests as database queries and 

so to treat message subscription requests as database 
data . 

[0059] Such an approach is counterintuitive as a mes- 
sage subscription inherently comprises the semantics 
of an "inquiry", a "filter", "question" and therefore a 
55 "query" for a specific type of information, a message. A 
message on the other hand inherently is of the nature of 
a data element, as it represents this type of information 
the subscriber is interested in and which will be sent to 
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the subscriber. 

[0060] The current teaching is depicted in Figure 3. 
According the current teaching, the messaging system 
or message broker comprises a database schema 
which reflects the possible types of subscriptions which s 
can be requested from the message broker. This data- 
base schema may depend on the message types pub- 
lished to the broker. Each subscription request received 
by the broker is transformed into a collection of tuples 
and stored in the database as database data (301). 10 
Without loss of generality one may exploit an underlying 
relational database being one of the most popular data- 
base concepts. Nevertheless the invention may also be 
based on other data models and database systems. 
Furthermore, the database must not be persistent, i.e. it is 
may be a main memory database in case no warehous- 
ing is required. Various mechanisms exist how to trans- 
form queries into tuples. As an example the teaching of 
F. Leymann, UDH: A universal relation system, Data & 
Knowledge Engineering 5 (1990), pp. 21 - 38 may be 20 2. 
exploited suggesting for the individual predicates a lin- 
ear representation comprising the search conjunctions 
and tuple conjunctions. This invention does only 
assume the existence of one transformation technique, 
which can be applied, and does not depend on a special 25 
one chosen. 3. 
[0061] Whenever a message is published, the broker 
transforms the message into a query (302). This query 
is performed on the subscription database and its 
answer set is the collection of subscriptions under which 30 
the message qualifies (303). The applications having 
made the retrieved subscriptions are identified and 
receive the message in the requested format 4. 



4.4 Value of the Current Approach 



35 



[0062] The current invention reduces subscription 
processing as defined in these messaging environ- 
ments to query processing known since many years 
from the field of databases. 40 5. 

[0063] Existing query techniques, implementations, 
and database systems can be used to perform pub- 
lish/subscribe in messaging system and message bro- 
ker environments. For a broad spectrum of business 
problems, no new algorithms must be invented and 45 
existing implementations can be reused. In warehous- 
ing environments this teaching is much more efficient 6. 
than the implementations being done without query 
processing technology in mind. 

50 

Claims 



tude of message-subscription-requests com- 
prising a query on a message qualifying under 
said query, 

said messaging system being characterized by 

comprising a database (301), and 

wherein said second receiving means treating 
said message-subscription-request as data- 
base data and storing said message-subscrip- 
tion-request to said database (301), and 

wherein said first receiving means treating said 
message-publication-request as a database- 
query (302) and retrieving from said database 
a collection (303) of message-subscription- 
requests qualifying under said database-query. 

Messaging system according to claim 1 , 

wherein said second receiving means trans- 
forming said message-subscription-request 
into a database data representation. 

Messaging system according to any of above 
claims, 

wherein said first receiving means transforming 
said message-publication-request into a data- 
base-query. 

Messaging system according to any of above 
claims, 

wherein said first receiving means store said 
message into said database or into a separate 
message-database. 

Messaging system according to any of above 
claims, 

wherein said messaging system is a message 
broker and/or wherein said database is part of 
a message warehouse. 

Messaging system according to any of above 
claims. 

wherein said database is stored in main mem- 
ory and/or on persistent memory. 



A messaging system comprising first receiving 
means for receiving a message-publication-request 
comprising a message to be published, and 55 

said messaging system comprising a second 
receiving means for receiving one or a multi- 



7. Messaging system according to claim 6, 

wherein said database is a relational database 
or a hierarchical database or a network data- 
base or an object-oriented database or a col- 
lection of flat files. 
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8. Messaging system according to any of above 
claims, 

further comprising sending means for sending 
said message to subscribers of said collection s 
of message-subscription-requests. 

9. A method of processing a message-publication- 
request comprising a message to be published, and 

10 

one or a multitude of message-subscription- 
requests comprising a query on a message 
qualifying under said query, 

said method comprising the following steps: is 

a first step of treating said message-sub- 
scription-request as database data and 
storing said message-subscription-request 
to said database (301), and 20 

a second step of treating said message- 
publication-request as a database-query 
(302) and retrieving from said database a 
collection (303) of message-subscription- 25 
requests qualifying under said database- 
query. 



15. Method according to claim 1 4, 

wherein said database is a relational database 
or a hierarchical database or a network data- 
base or an object-oriented database or a col- 
lection of flat files. 

16. Method according to any of claims 9 to 15. 

further comprising a third step of sending said 
message to subscribers of said collection of 
message-subscription-requests. 



1 0. Method according to claim 1 , 

30 

wherein said first step is preceded by a sub- 
scription-transformation-step wherein said 
message-subscription-request is transformed 
into a database data representation. 

35 

1 1 . Method system according to claim 9 or 10, 

wherein said second step is preceded by a 
message-transformation-step wherein said 
message-publication-request is transformed 40 
into a database-query. 



1 2. Method according to any of claims 9 to 1 1 , 

wherein said first step stores said message into as 
said database or into a separate message- 
database. 



13. Method according to any of claim 9 to 12, 

so 

wherein said method is executed by a message 
broker and/or wherein said database is part of 
a message warehouse. 



14. Method according to any of claims 9 to 13, ss 

storing said database in main memory and/or 
on persistent memory. 
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